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(1) Real Party in Interest ; 

The real party in interest is assignee Educational Testing Service. 

(2) Related Appeals and Interferences: 

No other appeals or interferences exist which relate to the present application or 
appeal. 

(3) Status of Claims: 

Claims 1, 4, 7-12, 16, and 21-26 are pending, finally rejected, and appealed herein. 
Claims 2-3, 5-6, 13-15, 17-20, and 27-30 were canceled, and claims 31-54 were withdrawn. 

(4) Status of Amendments: 

Amendments made to claims 1, 12, 16, and 26 in the amendment after final 
submitted September 18, 2008 were not entered by the Examiner. 

(5) Summary of Claimed Subject Matter: 

As an initial matter, it is noted that according to the Patent Office, the concise 
explanations under this section are for Board convenience, and do not supersede what the 
claims actually state, 69 Fed. Reg. 155 (August 2004), see page 49976. Accordingly, nothing 
in this Section should be construed as an estoppel that limits the actual claim language. 

As described in the abstract, for example, embodiments of the invention generally 
pertain to automatic evaluation of essays, including detection of overly repetitive word 
usage. More specificaUy, embodiments of the invention identify a word in the essay, and 
determine at least one feature associated with the word. A probability of the word being 
used in an overly repetitive manner is determined by mapping the feature to a model, the 
model having been previously generated by a machine learning application based on at 
least one human-evaluated essay. When the probability exceeds a threshold, embodiments 
of the invention annotate the essay to indicate the word is used in an overly repetitive 
manner. 

Claim 1 teaches a method for automatically evaluating an essay to detect a writing 
style error, excluding misspellings, that comprises electronically receiving an essay on a 
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computer system (page 8 line 18), and assigning a feature value automatically calculated 
(page 9 Une 14, page 19 line 1) for each feature of essay text segments (excluding essay 
function words (page 9 lines 17-24, page 11 lines 1-2, page 14 line 9 and 12-15, page 18 line 
20)). Further, the feature values are stored on a data storage device accessible by the 
computer system (page 21 line 14). Next, the feature values are compared with a model 
configured to identify writing style errors (page 17 lines 9-12). The model includes a 
decision tree to determine a probabUity associated with a likelihood of a writing style error, 
and the decision tree is generated based on a human evaluated essay (page 16 lines 13-16 
and 21-23). Finally, the method displays an indication of an identified writing style error 
(page 3 lines 23-24 and page 20 line 7). 

Claim 16 teaches a system for automatically evaluating an essay to detect a writing 
style error, excluding misspellings, that comprises a computer system configured to 
electronically receive an essay (page 18 line 18), and a feature extractor configured to 
assign a feature value (page 9 line 14, page 19 line 1) for each feature for essay text 
segments (excluding essay function words (page 9 lines 17-24, page 11 lines 1-2, page 14 line 
9 and 12-15, page 18 line 20)). Further, a data storage device is connected to the computer 
system and configured to store the feature values for the text segments (page 21 line 14). 
Also, a feature analyzer evaluates the essay for a writing style error by comparing the 
feature values for each text segment with a model generated using a human evaluated essay 
(page 17 lines 9-12). The model includes a decision tree to determine a probability 
associated with a likelihood of the writing style error, and the decision tree is generated 
based on a human evaluated essay (page 16 lines 13-16 and 21-23). Finally, a display 
presents the evaluated essay that includes an indication of an identified writing style error 
(page 3 lines 23-24 and page 20 line 7). 

(6) Grounds of Reiection to be Reviewed on Appeal: 

(a) Whether claims 1, 4, 7-10, 12, 16, and 21-24 and 26 have been properly rejected under 35 
U.S.C. § 103(a) as unpatentable over US. Patent No. 6,356,864 (Foltz) in view of U.S. 
Published Application No. 2004/0002994 (Brill), U.S. Published Application No. 
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2004/0093567 (Schabes), and, effectively, U.S. Published Application No. 2003/0149692 
(Mitchell). 

(b) Whether claims 11 and 25 have been properly rejected under 35 U.S.C. § 103(a) as 
unpatentable over Foltz in view of Brill and Schabes and further in view ofMitchelL 

(7) Argument; 

(a) Claims 1, 4, 7-10, 12, 16, and 21-24 and 26: 

It is noted that according to tlie Patent Office, a new ground of rejection in an 
Examiner's answer should be "rare" and should be levied only in response to such things 
as newly presented arguments by an AppUcant or to address a claim that the Examiner 
previously failed to address, 69 Fed. Reg. 155 (August 2004), see, e.g., pages 49963 and 
49980. Furthermore, a new ground of rejection must be approved by the Technology 
Center Director or designee and in any case must come accompanied with the initials of the 
conferees of the appeal conference, id., page 49979. AppeUants note that the SPE signed off 
on the final rejections. Accordingly it is not expected that reopening of prosecution will 
occur, since the SPE has already had the chance to consider the gravamen of the arguments 
below and has rejected them. 

Appellants respectfully submit that none of the prior art cited by the Examiner 
discloses, teaches, or suggests all the elements of the appealed claims. Therefore, neither 
independent claim 1 nor independent claim 16 is obvious based upon any of the cited art. 
Because claims 1 and 16 are patentable, their dependent claims are patentable at least as 
dependent from patentable base claims, per MPEP § 2143.03 and In re Fine, 837 F.2d 1071 
(Fed. Cir. 1988). Dependent claims 11 and 25 are addressed separately below. The 
dependent claims are further patentable based on the additional limitations that are added 
by each such dependent claim. Accordingly, Appellants respectfuUy assert that the 
rejections are improper and should be reversed. 

Limitations added to the independent claims in the amendment after final filed on 
September 18, 2008 were not entered by the Examiner. Appellants had amended claims 1 
and 16 to more clearly define the invention, e.g. to denote that function words of the essay 
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are to be excluded when the embodiments of the invention are calculating the feature 
values. These further limitations were originally in dependent claim 5, which had been 
previously rejected based on all four of the cited prior art references, Foltz, Brill, Schabes, 
and Mitchell. Appellants assert the Examiner's decision not to enter the amendment after 
final constitutes an assertion as detailed in the advisory action that the content of claim 5 
(now moved into independent claims 1 and 16) is still rejected based on all four of the prior 
art references, and appeal accordingly. 

AppeUants note that Mitchell extracts nouns, verbs, modifiers, prepositions, 
adjectives, and adverbs for example, and then submits data representations of such 
constituent parts of each student answer to semantic analysis. See for example paragraphs 
[0026] and [0048]-[0054] of Mitchell. Mitchell not only fails to teach or suggest but 
actually teaches away from not considering the function words of the essay, as taught and 
presently claimed in the present invention. In paragraph [0031] of the pubUshed version of 
the present patent application (U.S. Patent Application Publication 2004/0194036), 
corresponding to paragraph [0030] on page 9 of the as-ffled appUcation, the inventors state 
"Prior to generating the vector files, function words such as prepositions, articles, and 
auxiliary verbs, may be removed . For example, the function words (the, that, what, a, an, 
and, not) have been empirically found to increase the complexity of the analysis without 
contributing to the reliability of the result." (emphasis added). 

The Examiner's assertion that in Mitchell "certain words of the essay are not 
considered by the assessment tool since they are altered to the reduced variant form" is not 
entirely correct; such words are converted to the reduced variant form specifically so they 
can indeed be considered by Mitchell, in contrast to the claimed embodiments of the 
present invention which intentionally do not consider them. Appellants amended the 
independent claims in the (unentered) amendment after final to clarify that functional 
words are excluded from consideration by embodiments of the present invention, to 
distinguish such exclusion from merely altering words to a root form that then is 
considered. 

(h) Claims 11 and 25: 
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Dependent claims 11 and 25 warrant particular discussion and separate appellate 
review. As with the other claims previously described, Appellants respectfully disagree 
with the Examiner's assertion that Mitchell cures the shortcomings of the other cited 
references. AppeUants assert that Mitchell does not process pronouns (vs. general nouns, 
which are distinct from pronouns) at all, as taught and claimed by the present invention. 
Appellants cite for example paragraph [0015] of Mitchell in toto: "Preferably, the natural 
language processing parses the mark scheme answer into constituent parts such as nouns, 
verbs, adjectives, adverbs, modifiers and propositions." Appellants note that not only this 
paragraph but all of Mitchell entirely fails to mention "pronouns". 

In summary, as the various embodiments of the present invention include features 
neither taught nor suggested by the cited prior art references, either separately or together, 
the obviousness rejections cannot be sustained. Mere untaught possibilities are insufficient 
to defeat patentability, and a prior art reference must be considered in its entirety, i.e. as a 
whole. W.L. Gore & Associates, Inc. v. Garlock, Inc., Ill F.2d 1540, 220 USPQ 303 (Fed. 
Cir. 1983), cert, denied, 469 U.S. 851 (1984). Therefore, reversal is respectfuOy requested. 

Conclusion 

For the reasons advanced above. Appellants assert the rejected claims are indeed 
patentable, thus the rejections should be reversed. 

Respectfully submitted, 
Mayer Brown LLP 

By: J> . /^I^,,^ 

Marc D. McSwain 
Registration No. 44,929 
Direct No. (650) 331-2048 

Date: November 6, 2008 

Customer Number 26565 
MAYER BROWN LLP 
P.O. Box 2828 
Chicago, IL 60690-2828 
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APPENDIX A - APPEALED CLAIMS 

1. A method for automatically evaluating an essay to detect at least one writing style error, 

excluding misspellings, comprising: 

electronically receiving an essay on a computer system; 

assigning a feature value for each of one or more features for one or more text segments in 
the essay, wherein the feature values are automatically calculated by the computer 
system and wherein function words of the essay are excluded when calculating the 
feature values; 

storing the feature values for the one or more text segments on a data storage device 
accessible by the computer system; 

comparing the feature values for each text segment with a model configured to identify at 
least one writing style error, wherein the model includes at least one decision tree to 
determine a probability associated with a likelihood of the at least one writing style 
error, and wherein the at least one decision tree is generated based on at least one 
human evaluated essay; and 

displaying an indication of an identified writing style error. 

4. The method of claim 1 wherein the comparison step comprises extracting patterns from 
the feature values, wherein the patterns are based on the presence or absence of features 
associated with each word in the essay. 

7. The method of claim 1 wherein the feature values comprise the ratio of the evaluated text 
segment occurrences in the essay to the total number of text segments in the essay. 
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8. The method of claim 1 wherein the feature values comprise the average, over all 
paragraphs of the essay, of the ratio of the number of times the evaluated text segment 
occurs in a paragraph of the essay, over the total number of text segments in the paragraph. 

9. The method of claim 1 wherein the feature values comprise the largest value of the ratio 
of the number times the evaluated text segment occurs in a paragraph of the essay over the 
total number of text segments in the paragraph, wherein the ratio is calculated for each 
paragraph in the essay. 

10. The method of claim 1 wherein the feature values comprise the length, measured in 
characters, of the text segment. 

11. The method of claim 1 wherein the feature values comprise a value indicating whether 
the text segment includes a pronoun. 

12. The method of claim 1 wherein the feature values comprise a value representing the 
interval distance between consecutive text segment occurrences determined by calculating 
the number of intervening characters. 

16. A system for automatically evaluating an essay to detect at least one writing style error, 
excluding misspellings, comprising: 

a computer system configured to electronically receive an essay; 
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a feature extractor configured to assign a feature value for each of one or more features for 
one or more text segments in the essay wherein function words of the essay are 
excluded when calculating the feature values; 

a data storage device, connected to the computer system, configured to store the feature 
values for the one or more text segments; 

a feature analyzer configured to evaluate the essay for at least one writing style error by 
comparing the feature values for each of one or more text segments with a model 
generated using at least one human evaluated essay, wherein the model includes at 
least one decision tree to determine a probability associated with a likelihood of the 
at least one writing style error, and wherein the at least one decision tree is generated 
based on at least one human evaluated essay; and 

a display for presenting the evaluated essay, wherein the evaluated essay includes an 
indication of at least one identified writing style error. 

21. The system of claim 16 wherein the feature extractor comprises an essay ratio 
calculator configured to generate a value representing the ratio of the number of times the 
evaluated text segment occurs in the essay to the total number of text segments in the essay. 

22. The system of claim 16 wherein the feature extractor comprises an average paragraph 
ratio calculator configured to generate a value representing the average over all 
paragraphs in the essay of the ratio of the number of times the evaluated text segment 
occurs in a paragraph of the essay over the total number of text segments in the paragraph. 
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23. The system of claim 16 wherein the feature extractor comprises a highest paragraph 
ratio calculator configured to generate a value representing the largest ratio of the number 
of times the evaluated text segment occurs in a paragraph of the essay over the total 
number of text segments in the paragraph. 

24. The system of claim 16 wherein the feature extractor comprises a length calculator 
configured to generate a value representing the length, measured in characters, of the text 
segment. 

25. The system of claim 16 wherein the feature extractor comprises an identifier to 
determine whether the text segment includes a pronoun. 

26. The system of claim 16 wherein the feature extractor comprises a distance calculator 
configured to generate a value representing the distance measured in characters between 
consecutive text segment occurrences. 
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APPENDIX B - EVIDENCE 

None (this sheet made necessary by 69 Fed. Reg. 155 (August 2004), page 49978). 
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APPENDIX C - RELATED PROCEEDINGS 

None (this sheet made necessary by 69 Fed. Reg. 155 (August 2004), page 49978). 
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